Method for operating virtual reality glasses in a vehicle and virtual reality system with virtual reality glasses and a vehicle

ABSTRACT

A method of operating virtual reality glasses in a vehicle in which a risk of motion sickness for a wearer of the virtual reality glasses is reduced with the aid of the method. A virtual reality system (IO) includes the virtual reality glasses and the vehicle. According to the method, a vehicle movement of the vehicle is evaluated in such a way that ultimately, after the image data describing the virtual surroundings have been split into a background image dataset and a foreground image dataset, a lateral offset for a position of an object in the foreground in comparison with the background is determined, so that virtual surroundings which are processed in this way can be determined and displayed. Alternatively, the virtual surroundings can be enlarged in accordance with the vehicle movement, and processed virtual surroundings can be determined by means of a movement of the enlarged virtual surroundings along a movement trajectory, and displayed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. national stage of International ApplicationNo. PCT/EP2020/069769, filed on Jul. 13, 2020. The InternationalApplication claims the priority benefit of German Application No. 102019 124 386.6 filed on Sep. 11, 2019. Both the InternationalApplication and the German Application are incorporated by referenceherein in their entirety.

BACKGROUND

Described below is a method for operating virtual reality glasses in avehicle, in particular in a motor vehicle, as well as to a virtualreality system with virtual reality glasses and a vehicle.

In a virtual reality system, that is, a so-called virtual reality (VR)system, or a so-called mixed reality (MR) system, a wearer of a pair ofglasses, that is, a wearer of VR glasses or MR glasses, oftenexperiences symptoms of travel sickness if the wearer is wearing theglasses in moving surroundings. The reason for this is that there is acontradiction between the perceived sensory stimuli, that is, themacroscopic movement of the wearer's body, for example, in a movingmotor vehicle in which the wearer of the VR glasses is sitting, and thevirtual surroundings displayed by the glasses and visually perceived bythe wearer. This is because, while the wearer of the VR glasses issitting in the motor vehicle and is moving along with the movement ofthe motor vehicle, the wearer receives visual stimuli that deviate fromthis motion because of the glasses. Expressed in simple terms, thevisually perceived movement and the movement and acceleration felt bythe wearer's body do not match for the wearer of the glasses. This oftenleads to a strong feeling of being unwell, as the wearer suffers fromso-called travel sickness, which is often called “motion sickness” andis known as kinetosis in medical terminology.

The US patent 2018 0089900 A1 shows a VR system for motor vehicles forreducing the risk of kinetosis for a wearer of a corresponding pair ofVR glasses. For this purpose, the VR system provides a display in whichvisual cues correspond to the movement physically experienced by theperson. For example, visual cues are displayed that indicate a flowingmotion past the wearer. A speed of this visual information is adapted tothe speed or acceleration of the motor vehicle.

The US patent 2017/0253252 A1 shows a system in which a driver wears VRglasses in a motor vehicle. Depending on the movement of the vehicle, avisual graphic is displayed on the VR glasses which is in the form, forexample, of a force vector of an expected movement of the vehicle, whichin this case is driven autonomously. In addition, a virtual sphere canbe moved in a virtual container to indicate an expected direction ofmotion of the motor vehicle.

The US patent 2006/0015000 A1 describes a system for reducing akinetosis risk for an occupant of a moving motor vehicle, in which videodata of the vehicle's surroundings is acquired and displayed to theoccupant of the vehicle. A section or an enlarged section of the videoimage can be displayed, depending on which section of the image for theoccupant corresponds to an actual field of view of the occupant, basedon their viewing direction.

SUMMARY

Described below is a way of which a pair of virtual reality glasses canbe operated in such a way that the risk of kinetosis is reduced for awearer of the virtual reality glasses in a moving vehicle.

Existing known methods for reducing the risk of kinetosis for the wearerof virtual reality (VR) glasses in a moving vehicle, may be by way ofspecific contents created which are then displayed to the wearer. Forexample, content influenced by a current trajectory of the vehicle isproduced for the VR glasses, such as the sphere moving in a containerdescribed above. However, it is not always desirable to createadditional content for a virtual surroundings displayed by the VRglasses merely to reduce the risk of kinetosis, as such a procedure istime-consuming and can lead to unwanted virtual content in the virtualsurroundings displayed. The method described is therefore based on thefact that only already known image contents of the virtual surroundingsare processed without the need to explicitly generate new imagecontents, in order to contribute to a reduction of the kinetosis riskfor the wearer of the VR glasses.

The first aspect of the method of operating VR glasses in a vehiclereduces a risk of kinetosis for a wearer of the VR glasses in thevehicle. The vehicle may be a motor vehicle operating on ground orwater. Alternatively, for example, an aircraft can be used as thevehicle by operating the VR glasses. In the following, the example of amotor vehicle is always given as the vehicle in which the VR glasses areoperated. The VR glasses are generally used to display a virtualsurroundings described by predefined image data to the wearer of the VRglasses. VR refers to the representation and simultaneous perception ofa real event and its physical properties in real time in acomputer-generated interactive virtual environment. As an alternative tothe VR glasses described here, the method can be implemented using a VRcontact lens, for example. However, the following text always refers toa pair of VR glasses as the display device for displaying the virtualsurroundings.

In the method for operating a control device of the VR glasses, themovement data describing a vehicle movement, acquired by an acquisitiondevice of the vehicle, is received. For example, a vehicle sensor systemcan be used to quantify a speed, acceleration, angle of wheel impact,and/or a degree of vehicle vibration caused by a ride on a bumpy road.Alternatively, or in addition, predictive route data can be used asmovement data, which includes information about a previous route andwhich can therefore also be used to determine the aforementionedmovement data. Such predictive route data may be available, for example,to a vehicle's navigation system. This means that there is data thatdescribes the current movement and, if applicable, the expected futuremovement of the vehicle. This data is acquired in the individualacquisition devices and transferred to the control device of the VRglasses, for example, via a suitable communication link between anacquisition device of the vehicle. This communication link can beimplemented wirelessly, for example, via a wireless local area network(WLAN), a Bluetooth connection and/or a mobile data network based on thelong-term evolution (LTE), long-term evolution advanced (LTE-A), orFifth Generation (5G) mobile radio standard.

Next, the image data describing the virtual surroundings is split into abackground image dataset and a foreground image dataset. The backgroundimage dataset characterizes a background of the virtual surroundings.The foreground image dataset characterizes at least one objectpositioned in the foreground of the virtual surroundings. This splittingof the virtual surroundings is carried out by applying an imageprocessing criterion to the image data describing the virtualsurroundings. In the form of the image processing criterion, at leastone image processing algorithm is stored which enables the describedsplitting of the image data into the foreground image dataset and thebackground image dataset. For example, the image processing algorithmcan be based on edge detection, contrast detection, and/or artificialintelligence methods. For example, if a flat landscape with a mountainrange in the background is displayed as the virtual surroundings with asingle tree located in the foreground, the flat plane with the mountainrange as the background image dataset is separated from the tree as anobject positioned in the foreground of the virtual surroundings. As aresult, there are two planes of image data that have been separated fromeach other. If there are multiple objects in the foreground of thevirtual surroundings, they may be arranged on a common foreground plane.However, alternatively or in addition, it is possible to differentiatemultiple foreground planes.

Next, the acquired movement data is evaluated to determine a lateraloffset between a position of the at least one object positioned in theforeground of the virtual surroundings and the background. According tothe lateral offset, it can thus be specified that the foreground isshifted to the left relative to the background by, for example, onemillimeter towards the eyes of the wearer of the VR glasses. It isassumed here that the lateral offset is chosen in such a way that if theat least one object in the foreground is moved relative to thebackground of the virtual surroundings, the risk of kinetosis to thewearer of the VR glasses is reduced because a probability value for aprobability of kinetosis for the wearer of the VR glasses is below apredefined minimum probability value due to the relative displacement ofthe two image planes described. A direction of the lateral offsettypically corresponds to a direction of motion of the vehicle. Forexample, if the vehicle turns off to the right relative to a currentdirection of travel in a longitudinal direction of the vehicle, acorresponding lateral offset of the tree positioned in the foregroundrelative to the mountain range in the background is determined, in whichthe tree is moved from a current position to the left in the viewingdirection of the eyes of the wearer of the VR glasses. The two imageplanes are thus moved relative to each other in a manner that matchesthe received movement data of the vehicle. As an alternative to movingthe tree to the left, the mountain range can be moved from its currentposition to the right, or the tree is both moved to the left and themountain range to the right, i.e. the tree and the mountain range aremoved away from each other according to the lateral offset.

Next, to implement the relative offset, processed image data describinga processed virtual surroundings is determined, in which the at leastone object in the foreground is shifted relative to the backgroundaccording to the determined lateral offset. Thus, new image data isgenerated for display on the VR glasses, which data describes thedesired processed virtual surroundings due to the lateral offset.Lastly, the processed virtual surroundings described by the processedimage data is displayed using the VR glasses.

An advantage of the method is that existing VR content, i.e. the virtualsurroundings already described using the predefined image data, is usedwithout the need for virtual image content produced specifically toreduce the kinetosis risk of the wearer of the VR glasses. For example,a 360 degree representation of a virtual landscape, a video and/or acartoon can be displayed as the virtual surroundings. Based on themovement data describing the vehicle's movement, a correspondingrelative movement of the background plane with respect to the foregroundplane takes place, wherein such movement is also physically felt by thewearer of the VR glasses due to the movement of the vehicle. The visualmovement in the virtual surroundings is thus adapted to the movement ofthe vehicle experienced by the wearer. This significantly reduces therisk of kinetosis for the wearer of the VR glasses. This method is alsoapplicable to both static and moving virtual environmental contentdisplayed on the VR glasses. For example, it can also be used for adisplayed cartoon or video content, wherein a given object in theforeground of such moving video content is moved, for example, in such away that its movement specified by the video content is offset with thedetermined lateral offset, for example, by adding the two. This reducesthe risk of kinetosis for the wearer of VR glasses, and in aparticularly simple and convenient manner for the wearer, since noadditional image content is generated and displayed.

The described embodiments result in additional advantages.

In an advantageous embodiment, the determined lateral offset and asteering angle of the vehicle, which is included in the movement data,are correlated non-linearly. Therefore, in the event of a particularlylarge movement of the vehicle, the foreground is not shifted as muchrelative to the background as would be expected from the movement dataof the vehicle. Instead, it is possible to carry out only an attenuatedmovement of the foreground relative to the background, which ultimatelyonly implies the direction of the movement experienced by the wearer ofthe VR glasses due to the vehicle movement by the lateral offset of thetwo image planes with respect to each other. This is because such anattenuated movement in the virtual surroundings in comparison to themovement actually made by the vehicle can be sufficient to reduce thediscrepancy between the movement seen and felt by the wearer of the VRglasses in such a way that the risk of kinetosis is significantlyreduced. One advantage of this is that no sharp deviation or movementthat might be perceived as disturbing is artificially generated withinthe virtual surroundings, which is unlikely to be desirable for thewearer since it strongly influences the virtual surroundings, forexample. The attenuated relative movement of the foreground with respectto the background achieves a minimal change in the processed virtualsurroundings displayed compared with the original virtual surroundings,while at the same time significantly reducing the risk of kinetosis forthe wearer of the VR glasses. This ultimately keeps the effect of themethod on the processed virtual surroundings displayed to a minimum sothat the wearer of the VR glasses continues to be conveniently shown thevirtual surroundings he or she wants, since the processed virtualsurroundings differs as little as possible from the original virtualsurroundings.

According to a further embodiment, it is provided that the at least oneobject positioned in the foreground of the virtual surroundings isenlarged by a predefined magnification factor. In the processed virtualsurroundings, the enlarged at least one object is displaced relative tothe background. For example, it may be provided that the object detectedin the foreground in the form of the tree is displayed enlarged by afactor of ten percent, for example, with respect to the background andin the processed virtual surroundings is displayed as a tree enlarged byten percent and moved to the left by, for example, five millimeters inthe viewing direction of the wearer. This makes it possible for an imagegap, which generally arises in the background image dataset due todisplacement of the object in the foreground relative to the background,to be hidden and therefore not visible in the processed virtualsurroundings. This is because the enlargement of the object in theforeground does not create a gap visible to the wearer of the VRglasses, despite this lateral offset between the original position ofthe object in the foreground and the position now occupied by thisobject in the processed virtual surroundings provided, because this gapis already concealed by the object itself due to the enlargement of thevirtual object in the processed virtual surroundings. For example, themagnification factor used here can be one percent, two percent, threepercent, four percent, five percent, ten percent, 15 percent, 20percent, 25 percent, or 30 percent.

This makes it particularly simple to view the processed virtualsurroundings using VR glasses without processing the background imagedataset and simply by enlarging the virtual object in the virtualforeground. The wearer of the VR glasses thus does not perceive anypossibly disturbing effects of the method in the virtual surroundingscaused by the displacement of the object in the foreground relative tothe background. A suitable magnification of the at least one object inthe foreground can be chosen depending on the size of the lateraloffset. The two values of the lateral offset and the predefinedmagnification factor can thus be correlated.

According to an additional embodiment it is provided that, in case animage gap arises in the processed virtual surroundings due to thedisplacement of the at least one object in the foreground according tothe determined lateral offset, the image gap in the processed virtualsurroundings is closed with a virtual image fill content by applying animage gap fill criterion to the processed image data. The image gap fillcriterion thus contains information about, for example, a coloring inthe surroundings of the object in the foreground so that, for example,the virtual image gap can be filled with the corresponding virtual imagefill content by a content-based image fill, for example, using aso-called content-aware fill algorithm which is stored in the image gapfill criterion. Thus, artificially created content is used to close theimage gap that has arisen, so that when the at least one object isdisplaced in the foreground relative to the background, even withoutenlargement of the at least one object in the foreground, no visibleempty space appears in the background. The image gap fill criterion cantherefore be used to determine a suitable fill content based on thecolor values of the background image. This enables the wearer of the VRglasses to enjoy a trouble-free and comfortable experience of theprocessed virtual surroundings using the VR glasses, making minimal useof artificially generated image content.

In an additional embodiment, the acquired movement data is evaluated inorder to determine a rotation angle of the at least one objectpositioned in the foreground of the virtual surroundings relative to thebackground. Processed image data is then determined that describes theprocessed virtual surroundings in which the at least one object in theforeground is shifted relative to the background according to thedetermined lateral offset and additionally rotated according to thedetermined rotation angles. In the case of the relative rotation of atleast one object in the foreground relative to the background of thevirtual surroundings, the probability value for the probability of akinetosis for the wearer of the VR glasses is reduced below thepredefined minimum probability value, so that the risk of kinetosis forthe wearer of the VR glasses is also reduced. The rotation angle that isdetermined also reflects the movement performed by the vehicle, which isbased on the movement data. For example, in addition to a purely lateraldisplacement, a slight rotational movement can be performed by thevirtual object in the virtual foreground in order to visually displaythe movement experienced by the wearer of the VR glasses to the weareras a result of the driving maneuver of the vehicle, for example, whenthe vehicle is cornering, in a realistic manner using the processedvirtual surroundings. Thus, during a right-hand turn (relative to thedirection of travel in the longitudinal direction of the vehicle), inaddition to the purely lateral displacement to the left in the viewingdirection, the slight turning movement counter-clockwise experienced bythe body of the wearer can be simulated by turning the at least oneobject in the foreground by, for example, five degrees counter-clockwise(a rotation axis in this case passes through a center of the at leastone virtual object in the viewing direction of the wearer). This furtherincreases the comfort for the wearer of the VR glasses, as such amovement of the two image planes relative to each other has aparticularly beneficial effect in terms of reducing the risk ofkinetosis while the VR glasses are worn when driving with the vehicle.

Lateral displacement can generally be understood to mean not only amovement along a transverse axis of the vehicle, i.e. to the left orright with respect to a direction of travel in the longitudinaldirection of the vehicle, but also as a movement perpendicular theretoupwards or downwards, i.e. along a vertical vehicle direction. Forexample, if the vehicle is traveling on a bumpy road with numerouspotholes, which causes the wearer of the VR glasses to experience amovement upwards or downwards in the vehicle's vertical direction, thismovement can also be simulated by a corresponding lateral displacementof the object positioned in the foreground relative to the background.

A second aspect of the method of operating VR glasses in a vehicle alsoreduces a risk of kinetosis for a wearer of the VR glasses in thevehicle. In this method for operating a control device of the VRglasses, data is received about the surroundings acquired by anacquisition device of the vehicle that describes a vehicle movement ofthe vehicle, as has already been explained for the first aspect of themethod. However, next the virtual surroundings are enlarged by apredefined magnification factor so that an edge region of the virtualsurroundings is outside a display area displayed on the VR glasses. Thedisplay area includes at least one sub-region of a field of vision of awearer of the VR glasses. For example, the virtual surroundings can beenlarged by ten percent, resulting in the border of the virtualsurroundings being no longer displayed by the VR glasses because itextends beyond their display area. The magnification may be chosendepending on the image quality and resolution of the VR glasses. Forexample, the magnification factor used here can be one percent, twopercent, three percent, four percent, five percent, ten percent, 15percent, 20 percent, 25 percent, or 30 percent.

The display area shown includes at least one predefined sub-region ofthe field of vision of the wearer of the VR glasses. This is often thecase, for example, with a video or a cartoon as the virtualsurroundings, which is displayed, for example, on a virtual canvas inthe field of vision of the wearer of the VR glasses. If this is thecase, the virtual surroundings displayed on the virtual canvas isenlarged while maintaining a constant size of the canvas, so that anedge region not displayed on the VR glasses is formed around the virtualcanvas. In this example, the display area is the virtual canvas. Ingeneral, the viewing area may be considered to be the area displayed onthe VR glasses for the VR glasses wearer, in which the virtualsurroundings are displayed. The viewing area may not extend across theentire field of vision of the VR glasses wearer. The display area thusmay include only one sub-region of the field of vision of the VR glasseswearer.

Next, the acquired movement data is evaluated to determine a movementtrajectory for at least one area of the enlarged virtual surroundingslocated in the display area displayed. Ultimately, this determines atrajectory for any pixel in the enlarged virtual surroundings, whichleads from a current position of that pixel on the display area to atarget position on the display area. When the enlarged virtualsurroundings is moved according to the determined movement trajectory,the risk of kinetosis for the VR glasses wearer is reduced in such a waythat a probability value for a probability of a kinetosis is below apredefined minimum probability value. For example, if a central objectis present in the enlarged virtual surroundings, it can be moved usingthe movement trajectory from a current position of a center of theobject in the enlarged virtual surroundings to a position, for example,five millimeters to the left of this virtual object in the viewingdirection of the wearer, wherein the entire enlarged virtualsurroundings is also moved according to the described movementtrajectory (i.e. not only the object named by way of example). Thisresults in processed image data being determined as part of the method,wherein the processed image data describes a processed virtualsurroundings in which the enlarged virtual surroundings is movedaccording to the movement trajectory. The processed virtual surroundingsdescribed by the processed image data is then displayed on the VRglasses.

By first enlarging the virtual surroundings, despite the movement of theimage according to the movement trajectory it is possible to fill theentire display area with the virtual surroundings, because an area ofthe enlarged virtual surroundings always moves into view from thepreviously hidden edge region if, for example, the processed virtualsurroundings is displaced in a specific direction according to themovement trajectory. Thus, the entire contents of the display area ofthe VR glasses can ultimately be displaced according to the movementdata of the vehicle. For this purpose, the virtual surroundings isfirstly enlarged and the edge region of the enlarged virtualsurroundings is defined by limiting the display area of the VR glassesto the sub-region of the field of vision of the VR glasses wearer. Dueto the edge regions protruding outside of this inner display area, imagecontents of the virtual plane are available in all directions around thedisplay area, which can be moved into the visible display area when theimage content is shifted in the visible region. This creates theimpression of a stationary viewing window when the entire contents areshifted. Thus, with a stationary display area, the enlargement of thevirtual surroundings ultimately ensures that the entire virtualsurroundings can be moved according to the movement trajectory withoutcausing an image gap to appear in the displayed virtual surroundings inthe edge region of the display area. The impression perceived by the VRglasses wearer ultimately corresponds to a kind of camera panning orcamera movement.

The advantage of this method is that most of the currently availableimage material is in the form of two-dimensional images, which generallydo not fill the 360-degree viewing angle available in VR glasses. Thiswill probably even continue to be the case, as a constant head rotationof full 360-degree angles is usually not attractive to the VR glasseswearer. This proposal of movement alignment using the determinedmovement trajectory does not require a complex algorithm for planeextraction or plane creation and, where applicable, image filling.Simply enlarging and shifting the content of the virtual surroundings issufficient to visually support any movement experienced by the wearer.

At present, due to hardware limitations of VR glasses, furtheradvantages are also obtained, as there is usually high-resolution imagematerial available while the VR glasses themselves have a lowerresolution. Due to the often much higher resolution of the image contentof the virtual surroundings, digital magnification in the VR glassesdoes not mean a loss of image quality for the wearer of the VR glasses,while at the same time the risk of kinetosis can be significantlyreduced. Thus, a particularly convenient reduction of the risk ofkinetosis is achieved for the wearer of the VR glasses.

In an advantageous embodiment of the method, a distance traveled by theregion of the enlarged virtual surroundings in the displayed displayarea according to the determined movement trajectory and a steeringangle of the vehicle, which is included in the movement data, arecorrelated non-linearly. For example, the distance between the currentposition of a specific pixel and its target position as a result of thespecified movement trajectory may thus be shorter than would be expectedbased on the detected steering angle of the vehicle. Thus, analogous tothe non-linear correlation between the determined lateral offset and thesteering angle of the vehicle described above in connection with thefirst aspect of the method, the movement trajectory in itself may covera shorter distance than would be assumed based on the movement of thevehicle, since a larger movement of the vehicle relative to a firstmovement of the vehicle need not result in a larger change of positionof the virtual surroundings compared to the change of position in thefirst movement. This means that although the risk of kinetosis isreduced, there is no noticeable adverse impact on the virtualsurroundings itself due to the described method.

In an additional embodiment, when displaying the processed virtualsurroundings described by the processed image data on the virtualreality glasses the processed virtual surroundings always fills theentire display area. Therefore, it is explicitly not provided that, dueto the movement along the movement trajectory, the virtual surroundingsis shifted in such a way that a gap in the displayed virtualsurroundings is ultimately created despite the presence of the edgeregion. For this reason, it is provided that the entire display area ofthe virtual surroundings on the VR glasses is always filled by at leastone sub-region of the enlarged virtual surroundings. This prevents gapsthat suddenly appear in the edge region of the display area of thevirtual surroundings from causing irritation to the VR glasses wearer,which allows a meaningful use of the VR glasses in the moving vehicle.

A particularly advantageous embodiment of the method provides that theVR glasses includes an acquisition unit and the control device takesinto account glasses movement data, acquired by the acquisition unit,that describes a movement of the VR glasses as movement data. Inaddition to the movement data described above, which describes theacquired vehicle movement of the vehicle, data provided by the VRglasses themselves may be taken into account to determine the lateraloffset and/or the movement trajectory definitively. For example, headmovements of the VR glasses wearer can be detected, which can amplify orcounteract, for example, a movement for the wearer in a direction ofmotion of the vehicle. For example, a movement, a speed of movement, arotation and/or an acceleration of the movement of the glasses can beacquired and provided as the glasses movement data. For this purpose,the VR glasses include an appropriate glasses sensor device as anacquisition unit. As a result, a particularly reliable reduction of therisk of kinetosis for the VR glasses wearer can be achieved, since notonly can the external influences on his or her body be taken intoaccount, but the actual movement of the VR glasses is also taken intoaccount by the glasses movement data.

The virtual reality system includes VR glasses and a vehicle. The VRsystem is designed to carry out a method as described above based on theevaluation of the determined lateral offset, and/or a method asdescribed above based on the determination of a movement trajectory. Thetwo methods described can therefore ultimately be combined. Theembodiments presented in connection with the two methods and theiradvantages apply accordingly, where applicable, to the VR system. Forthis reason, the corresponding further developments of the virtualreality system are not described again here.

The vehicle may be designed as a motor vehicle, in particular as apassenger car or heavy goods vehicle, or as a passenger bus ormotorcycle.

As an alternative to the VR glasses described above, corresponding mixedreality (MR) glasses can be used.

The control device of the VR glasses has a processor device which isconfigured to carry out an embodiment of the method according to theinvention. For this purpose, the processor device may include at leastone microprocessor and/or at least one microcontroller and/or at leastone FPGA (field programmable gate array) and/or at least one DSP(digital signal processor). Furthermore, the processor device can haveprogram code which is configured, when executed by the processor device,to carry out the embodiment of the method according to the invention.The program code can be stored in a data memory of the processor device.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects and advantages will become more apparent andmore readily appreciated from the following description of the exemplaryembodiments, taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a schematic representation of a virtual reality system for avehicle,

FIG. 2 is a schematic representation of a signal flow graph for a methodfor operating virtual reality glasses in a vehicle, taking into accounta lateral offset; and

FIG. 3 is a schematic representation of a signal flow graph for a methodfor operating virtual reality glasses in a vehicle, taking into accounta determined movement trajectory.

DETAILED DESCRIPTION

In the exemplary embodiments, the described components of theembodiments each represent individual features, which are to beconsidered independently of each other and be developed independently ofeach other. Therefore, the disclosure is also intended to includecombinations of the features of the embodiments other than thosepresented. Furthermore, the embodiments described can also be extendedto include other features already described.

In the figures, identical reference signs designate functionallyequivalent elements.

FIG. 1 shows a sketch of a virtual reality (VR) system 10. The VR system10 includes a pair of VR glasses 12 and a vehicle 14, which is a motorvehicle. A wearer 16 of the VR glasses 12 is present in the vehicle 14.The VR glasses 12 includes a control device 18 which is used to controlthe VR glasses 12. The vehicle 14 includes an acquisition device 20which acquires movement data describing a vehicle movement of thevehicle 14. The acquired movement data is transmitted by the acquisitiondevice 20 to the control device 18 of the VR glasses 12. In S1, themovement data describing the movement of the vehicle 14, acquired by theacquisition device 20 of the vehicle 14, is received by the controldevice 18 of the VR glasses 12. The acquisition device 20 acquires acurrent acceleration, a current speed, a current steering angle and/or acurrent change in altitude of the vehicle 14, for example due to drivingon a bumpy road.

FIG. 2 shows a sketch of a virtual surroundings 22 that is displayed bythe VR glasses 12. The virtual surroundings 22 in this case is stored inthe control unit 18 based on predefined image data. The virtualsurroundings 22 shows a landscape, in the background 24 of which amountain range can be seen, whereas in a foreground 26 of the virtualsurroundings 22 a tree 28 can be seen as an object 28 positioned there.In S2, the image data describing the virtual surroundings 22 is splitinto a background image dataset characterizing the background 24 of thevirtual surroundings 22, and at least one foreground image datasetcharacterizing at least one tree 28 positioned in the foreground 26 ofthe virtual surroundings 22. This splitting is carried out by applyingan image processing criterion to the image data describing the virtualsurroundings 22. For example, classical methods of digital imageprocessing are thus used, so that, for example, using an edge filterand/or a sharpness criterion, the tree 28 can be distinguished from thebackground 24 as an object 28 in the foreground 26. In the background 24there is now an image gap 30, since the tree 28, now described only bythe foreground image dataset, is cut out of the background imagedataset.

In S3, the acquired movement data of the vehicle 14 is evaluated todetermine a lateral offset 32 between the tree 28 positioned in theforeground 26 of the virtual surroundings 22 and the background 24. FIG.2 shows the lateral offset 32 in the form of an arrow. In addition,dashed lines are used to sketch the original position of the tree, whichis now occupied by the image gap 30 in the background image data set.

Then, in S4, processed image data is determined that describes aprocessed virtual surroundings 34 in which the tree 28 in the foreground26 is shifted relative to the background 24 according to the determinedlateral offset 32. Then, the virtual surroundings 34 described by theprocessed image data is displayed on the VR glasses 12 for the wearer16.

The presence of the lateral offset 32 ensures that, due to the relativedisplacement of the tree 28 in the foreground 26 with respect to thebackground 24 of the virtual surroundings 22, a kinetosis risk for thewearer 16 of the VR glasses 12 is reduced in such a way that aprobability value for a probability of kinetosis for the wearer 16 isbelow a predefined minimum probability value. For example, the risk ofsuffering from travel sickness, known as kinetosis, for the wearer 16 ofthe VR glasses 12 is significantly reduced despite the movement of thevehicle 14 according to the movement data.

If, as outlined here, an image gap 30 arises in the processed virtualsurroundings 34 due to the displacement of the tree 28 in the foreground26 according to the determined lateral offset 32, the image gap 30 inthe processed virtual surroundings 34 is closed with a virtual imagefill content 36 by applying an image gap fill criterion to the processedimage data. This image fill content 36 can be artificially generated,for example using a color value in an area surrounding the image gap 30.For example, if there is a virtual light green meadow around the virtualtree 28, a corresponding image fill content 36 in the same color as thisvirtual meadow can be used to ensure that the image gap 30 is no longervisible to the wearer 16 of the VR glasses 12.

Alternatively, or in addition to the lateral offset 32 described, thetree 28 in the foreground 26 can also be enlarged with a predefinedmagnification factor so that an enlarged tree 38 is visible in theforeground 26. In the processed virtual surroundings 34, the enlargedtree 38 is displaced relative to the background 24. This is illustratedhere by the differences between the original tree 28 represented withdashed lines and the enlarged tree 38 shown larger and drawn with asolid line in the processed virtual surroundings 34. That is to say, theinitial enlargement of the tree 28 by the predefined magnificationfactor, takes place in S5, after which the virtual surroundings 34 isdetermined in in an operation similar to S4.

The lateral offset 32 in this case can be correlated non-linearly withthe steering angle of the vehicle 14 included in the movement data. Inorder not to have to shift the respective image contents too far duringlarge vehicle movements, the displacement specified by the lateraloffset 32 can be carried out with an attenuated movement, whichultimately only implies the change in acceleration of the vehicle 14experienced by the wearer 16.

The acquired movement data can also be used to determine a rotationangle of the tree 28 in the foreground 26 relative to the background 24.This makes it possible, for example, to particularly emphasize aright-hand turn of the vehicle 14 by not only shifting the tree 28laterally, as in this example to the left in the viewing direction ofthe wearer 16, but also by slightly rotating it counter-clockwise (witha rotation axis parallel to the viewing direction). A rotation by anangle between 0 degrees and 90 degrees is possible.

FIG. 3 shows an alternative or additional method for reducing the riskof kinetosis for the wearer 16 of the VR glasses 12. After S1, asdescribed above, in this case the virtual surroundings 22 is enlarged bya predefined magnification factor so that an edge region 42 is formed ina display area 40 of the VR glasses 12, wherein parts of the enlargedvirtual surroundings 38 are still present in this edge region 42.However, the edge region 42 is no longer displayed on the VR glasses 12.Only the display area 40, the boundary of which with the edge region 42is sketched here with a dashed line, is displayed on the VR glasses 12.The enlargement of the virtual surroundings 22 to the enlarged virtualsurroundings 38 is carried out in S6. The display area 40 includes atleast one sub-region of a field of vision of the wearer 16 of the VRglasses 12. For example, the display area 40 with the virtual landscapedisplayed on it is located in the direct field of vision of the wearer16 of the VR glasses 12 as a canvas, with a visual edge region arrangedaround the display area 40 displayed in black and without any virtualcontent displayed there for the wearer 16. The visual edge region islocated around the edge of the display area 40, drawn with a solid line,and covers the content limited by a dashed and/or dotted line, since thecorresponding content outside the display area 40 is not displayed tothe wearer 16 of the VR glasses 12. A video display and/or a cartoonshown on the display area 40 is suitable as a virtual surroundings.

In S7, on the one hand, the acquired movement data is then evaluated todetermine a movement trajectory 44 for at least one area of the enlargedvirtual surroundings 38 located in the display area 40. Then, in S8processed image data describing a processed virtual surroundings 34 isdetermined, in which the enlarged virtual surroundings 38 is movedaccording to the movement trajectory 44. Finally, the processed virtualsurroundings 34 described by the processed image data is displayed onthe VR glasses 12. The distance traveled according to the determinedmovement trajectory 44 and the steering angle of the vehicle 14, whichis included in the movement data, can be correlated non-linearly. Inaddition, the processed virtual surroundings 34 always fills the wholedisplay area 40. The originally displayed display area 40 is marked herewith a dotted line, whereas the processed virtual surroundings 34displayed is shown within the dashed edge region and thus as displayarea 40′.

In addition to the described movement data, which describes the drivingmotion of the vehicle 14, glasses movement data of the VR glasses 12 canalso be used as movement data, which is provided by an appropriatesensor device of the VR glasses 12, the acquisition unit of the VRglasses.

Overall, the examples show a reduction of a kinetosis risk in a VRsystem 10. As an alternative to VR glasses, a pair of mixed realityglasses, i.e. a so-called MR system, can be provided.

A description has been provided with particular reference to preferredembodiments thereof and examples, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the claims which may include the phrase “at least one of A, B and C”as an alternative expression that means one or more of A, B and C may beused, contrary to the holding in Superguide v. DIRECTV, 358 F3d 870, 69USPQ2d 1865 (Fed. Cir. 2004).

1-10. (canceled)
 11. A method of operating virtual reality glasses in avehicle, in which a virtual surrounding described by image data isdisplayed by the virtual reality glasses, the method performed by acontrol device of the virtual reality glasses comprising: receivingmovement data describing a movement of the vehicle acquired by anacquisition device of the vehicle; splitting the image data describingthe virtual surrounding into a background image dataset thatcharacterizes a background of the virtual surrounding, and at least oneforeground image dataset that characterizes at least one objectpositioned in a foreground of the virtual surrounding, by applying animage processing criterion to the image data describing the virtualsurrounding; evaluating the movement data to determine a lateral offsetbetween a position of the at least one object positioned in theforeground of the virtual surrounding and the background; determiningprocessed image data describing a processed virtual surrounding in whichthe at least one object in the foreground is shifted relative to thebackground according to the determined lateral offset; and displayingthe processed virtual surrounding described by the processed image dataon the virtual reality glasses.
 12. The method as claimed in claim 11,wherein the determined lateral offset and a steering angle of thevehicle, included in the movement data, are correlated non-linearly. 13.The method as claimed in claim 12, further comprising enlarging the atleast one object positioned in the foreground of the virtual surroundingby a magnification factor, so that relative to the processed virtualsurrounding the enlarged at least one object is shifted relative to thebackground.
 14. The method as claimed in claim 13, wherein, when animage gap arises in the processed virtual surrounding due to adisplacement of the at least one object in the foreground according tothe determined lateral offset, the image gap in the processed virtualsurrounding is closed with a virtual image fill content by applying animage gap fill criterion to the processed image data.
 15. The method asclaimed in claim 14, further comprising evaluating the movement data todetermine a rotation angle of the at least one object positioned in theforeground of the virtual surrounding relative to the background, andwherein the processed image data is determined to describe the processedvirtual surrounding, in which the at least one object in the foregroundis shifted relative to the background according to the determinedlateral offset and rotated according to the determined rotation angle.16. The method as claimed in claim 11, further comprising enlarging theat least one object positioned in the foreground of the virtualsurrounding by a magnification factor, so that relative to the processedvirtual surrounding the enlarged at least one object is shifted relativeto the background.
 17. The method as claimed in claim 11, wherein, whenan image gap arises in the processed virtual surrounding due to adisplacement of the at least one object in the foreground according tothe determined lateral offset, the image gap in the processed virtualsurrounding is closed with a virtual image fill content by applying animage gap fill criterion to the processed image data.
 18. The method asclaimed in claim 11, further comprising evaluating the movement data isevaluated to determine a rotation angle of the at least one objectpositioned in the foreground of the virtual surrounding relative to thebackground, and wherein the processed image data is determined todescribe the processed virtual surrounding, in which the at least oneobject in the foreground is shifted relative to the background accordingto the determined lateral offset and rotated according to the determinedrotation angle.
 19. A method of operating virtual reality glasses in avehicle, in which a virtual surrounding described by image data isdisplayed by the virtual reality glasses, the method performed by acontrol device of the virtual reality glasses comprising: receivingmovement data describing a vehicle movement of the vehicle acquired byan acquisition device of the vehicle; enlarging the virtual surroundingby a magnification factor, so that an edge region of the virtualsurrounding lies outside a display area displayed on the virtual realityglasses, the display area including at least one sub-region of a fieldof view of a wearer of the virtual reality glasses; evaluating themovement data to determine a movement trajectory for at least one areaof the enlarged virtual surrounding located in the display area;determining processed image data describing a processed virtualsurrounding in which the enlarged virtual surrounding is moved accordingto the movement trajectory; and displaying the processed virtualsurrounding described by the processed image data on the virtual realityglasses, with a distance traveled by the region of the enlarged virtualsurrounding in the displayed display area according to the determinedmovement trajectory and a steering angle of the vehicle included in themovement data are correlated non-linearly.
 20. The method as claimed inclaim 19, wherein when displaying the processed virtual surroundingdescribed by the processed image data on the virtual reality glasses,the processed virtual surrounding fills the entire display area.
 21. Themethod as claimed in claim 20, wherein the virtual reality glassesinclude an acquisition unit and the control device takes into accountglasses movement data acquired by the acquisition unit that describes amovement of the virtual reality glasses as the movement data.
 22. Themethod as claimed in claim 19, wherein the virtual reality glassesinclude an acquisition unit and the control device takes into accountglasses movement data acquired by the acquisition unit that describes amovement of the virtual reality glasses as the movement data.
 23. Avirtual reality system operating in a vehicle, comprising: virtualreality glasses; and a controller of the virtual reality systemconfigured to perform a method including, receiving movement datadescribing a movement of the vehicle acquired by an acquisition deviceof the vehicle; splitting the image data describing the virtualsurrounding into a background image dataset that characterizes abackground of the virtual surrounding, and at least one foreground imagedataset that characterizes at least one object positioned in aforeground of the virtual surrounding, by applying an image processingcriterion to the image data describing the virtual surrounding;evaluating the movement data to determine a lateral offset between aposition of the at least one object positioned in the foreground of thevirtual surrounding and the background; determining processed image datadescribing a processed virtual surrounding in which the at least oneobject in the foreground is shifted relative to the background accordingto the determined lateral offset; and displaying the processed virtualsurrounding described by the processed image data on the virtual realityglasses.
 24. The virtual reality system as claimed in claim 23, whereinthe determined lateral offset and a steering angle of the vehicle,included in the movement data, are correlated non-linearly.
 25. Thevirtual reality system as claimed in claim 23, wherein the methodfurther comprising enlarging the at least one object positioned in theforeground of the virtual surrounding by a magnification factor, so thatrelative to the processed virtual surrounding the enlarged at least oneobject is shifted relative to the background.
 26. The virtual realitysystem as claimed in claim 23, wherein, when an image gap arises in theprocessed virtual surrounding due to a displacement of the at least oneobject in the foreground according to the determined lateral offset, theimage gap in the processed virtual surrounding is closed with a virtualimage fill content by applying an image gap fill criterion to theprocessed image data.
 27. The virtual reality system as claimed in claim23, further comprising evaluating the movement data is evaluated todetermine a rotation angle of the at least one object positioned in theforeground of the virtual surrounding relative to the background, andwherein the processed image data is determined to describe the processedvirtual surrounding, in which the at least one object in the foregroundis shifted relative to the background according to the determinedlateral offset and rotated according to the determined rotation angle.28. A virtual reality system operating in a vehicle, comprising: virtualreality glasses; and a controller of the virtual reality systemconfigured to perform a method including, receiving movement datadescribing a vehicle movement of the vehicle acquired by an acquisitiondevice of the vehicle; enlarging the virtual surrounding by amagnification factor, so that an edge region of the virtual surroundinglies outside a display area displayed on the virtual reality glasses,the display area including at least one sub-region of a field of view ofa wearer of the virtual reality glasses; evaluating the movement data todetermine a movement trajectory for at least one area of the enlargedvirtual surrounding located in the display area; determining processedimage data describing a processed virtual surrounding in which theenlarged virtual surrounding is moved according to the movementtrajectory; and displaying the processed virtual surrounding describedby the processed image data on the virtual reality glasses, with adistance traveled by the region of the enlarged virtual surrounding inthe displayed display area according to the determined movementtrajectory and a steering angle of the vehicle included in the movementdata are correlated non-linearly.
 29. The virtual reality system asclaimed in claim 28, wherein when displaying the processed virtualsurrounding described by the processed image data on the virtual realityglasses, the processed virtual surrounding fills the entire displayarea.
 30. The virtual reality system as claimed in claim 28, wherein thevirtual reality glasses include an acquisition unit, and wherein thecontroller takes into account glasses movement data acquired by theacquisition unit that describes a movement of the virtual realityglasses as the movement data.